Image processing device, image processing method, and image display system

ABSTRACT

An image processing device includes a memory that stores a program and a processor that executes the program stored in the memory to perform a process. The process includes obtaining input image data that is generated by combining, in a horizontal direction, an effective display area of first image data and an effective display area obtained by expanding an effective display area of second image data in the vertical direction, the vertical size of the second image data being smaller than the vertical size of the first image data; and generating second output image data corresponding to the second image data by reducing the vertical size of second intermediate image data among first intermediate image data and the second intermediate image data, which are obtained by dividing the input image data in the horizontal direction, based on the vertical size of the effective display area of the second image data.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation application filed under 35U.S.C. 111(a) claiming benefit under 35 U.S.C. 120 and 365(c) of PCTInternational Application No. PCT/JP2018/004314, filed on Feb. 8, 2018,the entire contents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to an image processing device, an imageprocessing method, and an image display system.

2. Description of the Related Art

There is a known image display system where multiple image data setswith different image sizes are displayed on displays with thecorresponding resolutions. For example, in an in-vehicle image displaysystem, image data sets generated by an image generating device aredisplayed on a center information display and a multi-informationdisplay, respectively.

In such an image display system, a controller (an image processingdevice) is provided for each display, and image processing correspondingto the characteristics of the display is performed on image data. Forthis reason, as the number of displays increases, the number ofcontrollers increases, which also increases costs and causes problemsin, for example, an installation space and the amount of heatgeneration.

WO 2004/066139 and Japanese Unexamined Patent Application PublicationNo. 2013-213859, for example, disclose an image display system wheremultiple image data sets are combined by an image generating device intoone image data set, and the one image data set is divided by acontroller into multiple image data sets so that the image data sets aredisplayed at the same frame rate. With this image display system, thenumber of controllers can be reduced.

However, to display multiple image data sets, which are obtained bydividing one image data set and have different sizes, at the same framerate as in the image display system disclosed in WO 2004/066139 andJapanese Unexamined Patent Application Publication No. 2013-213859, itis necessary to provide frame buffers in the controller. This is becauseit is necessary to temporarily store each frame of each image data setand output the frame at a pixel clock and a horizontal frequencycorresponding to the resolution of a display.

Here, to further reduce costs, reduce an installation space, and reducethe amount of heat generation in an image display system, line buffersmay be used in place of frame buffers. However, when line buffers areused, it is a prerequisite to match, for example, the horizontalfrequencies of image data sets before and after the image data sets arestored in the line buffers.

Therefore, for example, when image data sets with different verticalsizes are combined and displayed at the same or substantially the sameframe rate, the vertical blanking period becomes long with respect tothe effective display area of one of the image data sets with a smallersize. In this case, the display standard requirements are not satisfied,and an image is not displayed normally.

SUMMARY OF THE INVENTION

In an aspect of this disclosure, there is provided an image processingdevice including a memory that stores a program and a processor thatexecutes the program stored in the memory to perform a process. Theprocess includes obtaining input image data that is generated bycombining, in a horizontal direction, an effective display area of firstimage data and an effective display area obtained by expanding aneffective display area of second image data in the vertical direction,the vertical size of the second image data being smaller than thevertical size of the first image data; and generating second outputimage data corresponding to the second image data by reducing thevertical size of second intermediate image data among first intermediateimage data and the second intermediate image data, which are obtained bydividing the input image data in the horizontal direction, based on thevertical size of the effective display area of the second image data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a drawing illustrating an example of a system configuration ofan in-vehicle image display system;

FIG. 2 is a drawing illustrating an example of a layout of displays;

FIG. 3 is a drawing illustrating an example of a hardware configurationof an image processing device;

FIG. 4 is a drawing illustrating details of processes performed bydevices in an image display system according to a comparative example;

FIG. 5 is a drawing illustrating details of processes performed bydevices in an image display system according to a first embodiment;

FIG. 6 is a drawing illustrating an example of a functionalconfiguration of an image processor of an image processing deviceaccording to the first embodiment;

FIG. 7 is a flowchart illustrating image processing performed by theimage processor of the image processing device according to the firstembodiment;

FIG. 8 is a drawing illustrating details of processes performed bydevices in an image display system according to a second embodiment;

FIG. 9 is a drawing illustrating an example of a functionalconfiguration of an image processor of an image processing deviceaccording to the second embodiment;

FIG. 10 is a flowchart illustrating image processing performed by theimage processor of the image processing device according to the secondembodiment;

FIG. 11 is a drawing illustrating details of processes performed bydevices in an image display system according to a third embodiment;

FIG. 12 is a drawing illustrating an example of a functionalconfiguration of an image processor of an image processing deviceaccording to the third embodiment; and

FIG. 13 is a flowchart illustrating image processing performed by theimage processor of the image processing device according to the thirdembodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

An aspect of this disclosure makes it possible to reduce a blankingperiod in an image display system where image data sets obtained bydividing one image data set and having different sizes are displayed atthe same or substantially the same frame rate on multiple displays withdifferent sizes.

Embodiments of the present invention are described below with referenceto the accompanying drawings. In the specification and the drawings, thesame reference number is assigned to components having substantially thesame functional configuration, and repeated descriptions of thosecomponents are omitted.

First Embodiment

<System Configuration of in-Vehicle Image Display System>

First, a system configuration of an in-vehicle image display system isdescribed. FIG. 1 is a drawing illustrating an example of a systemconfiguration of an in-vehicle image display system.

As illustrated in FIG. 1, an image display system 140 installed in avehicle 100 includes an image generating device 110, an image processingdevice 120, a first display 131, and a second display 132.

The image generating device 110 is, for example, a navigation device ora head unit in an in-vehicle system, and includes an image generator 111(generating unit). The image generator 111 generates input image databased on first image data for the first display 131 and second imagedata for the second display 132, and outputs the input image data to theimage processing device 120.

For example, a semiconductor chip for executing image processing isprovided in the image processing device 120, and the image processingdevice 120 functions as an image processor 121 implemented by operationsof the semiconductor chip.

The image processor 121 obtains the input image data from the imagegenerating device 110 and divides the input image data in a horizontaldirection to generate first intermediate image data and secondintermediate image data.

Also, the image processor 121 generates first output image data andsecond output image data by reducing the vertical size of one or both ofthe first intermediate image data and the second intermediate imagedata. Further, the image processor 121 outputs the first output imagedata to the first display 131 and outputs the second output image datato the second display 132.

The first display 131 displays the first output image data output fromthe image processor 121. The first display 131 is assumed to have aresolution of, for example, 1920 pixels×1080 pixels.

The second display 132 displays the second output image data output fromthe image processor 121. The second display 132 is assumed to have aresolution of, for example, 1280 pixels×720 pixels.

<Example of Layout of Multiple Displays>

Next, an example of a layout of the first and second displays 131 and132 constituting the image display system 140 installed in the vehicle100 is described. FIG. 2 is a drawing illustrating an example of alayout of displays. As illustrated in FIG. 2, the first display 131 isdisposed in a center console 210 of the vehicle 100 and functions as acenter information display. The second display 132 is disposed in ameter panel 220 of the vehicle 100 and functions as a multi-informationdisplay.

<Hardware Configuration of Image Processing Device>

Next, a hardware configuration of the image processing device 120 isdescribed. FIG. 3 is a drawing illustrating an example of a hardwareconfiguration of an image processing device. As illustrated in FIG. 3,the image processing device 120 includes a host I/F 301, a DMAcontroller 302, and a command sequencer 303. These hardware componentsare connected to each other via a first bus 304.

Also, the image processing device 120 includes a flash 311 and a RAM/ROM312. These hardware components are connected to each other via a secondbus 313. The first bus 304 and the second bus 313 are connected to eachother via a bridge.

Further, the image processing device 120 includes a video capturer 321and a display control device 322, and the display control device 322 isconnected to the second bus 313.

The host I/F (interface) 301 is connected to an external device (notshown) and receives setting data transmitted from the external device.

The DMA (direct memory access) controller 302 performs a control tostore the setting data received by the host I/F 301 in the flash 311.The command sequencer 303 controls the entire image processing device120. The flash (flash memory) 311 stores the setting data received bythe host I/F 301. The RAM/ROM 312 functions as a main storage used bythe command sequencer 303 to control the entire image processing device120.

The video capturer 321 sequentially obtains the input image data outputfrom the image generating device 110 in units of multiple lines andoutputs the input image data to the display control device 322. Thedisplay control device 322 performs image processing according to thesetting data. The image processing device 120 functions as the imageprocessor 121 implemented by operations of the display control device322. With the above configuration, the image processing device 120 canobtain the input image data from the image generating device 110 andoutput the first output image data and the second output image data tothe first display 131 and the second display 132.

<Details of Processes Performed by Devices>

Next, details of processes performed by devices (here, the imagegenerating device and the image processing device) in the image displaysystem 140 are described. Below, details of processes performed bydevices in an image display system of a comparative example are firstdescribed, and then details of processes performed by devices in theimage display system 140 of the first embodiment are described.

(1) Details of Processes Performed by Devices in Image Display System

FIG. 4 is a drawing illustrating details of processes performed bydevices in an image display system according to a comparative example.Here, as illustrated in FIG. 4, the image display system is assumed tohave a configuration described below.

-   -   The image display system includes an image generating device        410, an image processing device 420, a first display 431, and a        second display 432.    -   The image generating device 410 generates image data 411 based        on image data (1920 pixels×1080 pixels) for the first display        431 and image data (1280 pixels×720 pixels) for the second        display 432. Also, the image generating device 410 outputs the        generated image data 411 as input image data 412 to the image        processing device 420.    -   The image processing device 420 divides the input image data 412        in a horizontal direction to generate first output image data        421 and second output image data 422.    -   The image processing device 420 does not include frame buffers,        but instead includes line buffers.    -   The first display 431 has a resolution of 1920 pixels×1080        pixels, and the second display 432 has a resolution of 1280        pixels×720 pixels.    -   The vertical frequency (refresh rate) of the input image data        412 is equal to the vertical frequency of the first output image        data 421. Also, the vertical frequency of the input image data        412 is also equal to the vertical frequency of the second output        image data 422.    -   The horizontal frequency of the input image data 412 is equal to        the horizontal frequency of the first output image data 421. The        horizontal frequency of the input image data 412 is also equal        to the horizontal frequency of the second output image data 422.

In this image display system, the devices perform processes describedbelow.

First, the image generating device 410 generates the image data 411 bycombining an effective display area 411-1 of the image data for thefirst display 431 and an effective display area 411-2 of the image datafor the second display 432 in the horizontal direction. For example, theeffective display area 411-1 is 1920 pixels×1080 pixels, and theeffective display area 411-2 is 1280 pixels×720 pixels.

Next, the image generating device 410 outputs the generated image data411 as the input image data 412 to the image processing device 420. As aresult, the image processing device 420 obtains the input image data412.

The image processing device 420 divides the input image data 412 in thehorizontal direction to generate the first output image data 421 and thesecond output image data 422.

In the first output image data 421, a horizontal blanking period and ahorizontal synchronization signal are set according to the resolution ofthe first display 431. Here, because the image processing device 420does not include frame buffers, the first output image data 421 issynchronized such that its vertical frequency becomes equal to thevertical frequency of the input image data 412. Therefore, the verticalblanking period of the first output image data 421 is set based on thevertical blanking period of the input image data 412, and thecorresponding vertical synchronization signal is set. The verticalblanking period of the first output image data 421 is preferably thesame as the vertical blanking period of the input image data 412.However, a difference of, for example, several lines is acceptable aslong as it is not noticeable on the display screen.

Accordingly, the effective display area 411-1 is displayed on the firstdisplay 431 at the same frame rate as the input image data 412.

On the other hand, in the second output image data 422, a horizontalblanking period and a horizontal synchronization signal are setaccording to the resolution of the second display 432. Also, because theimage processing device 420 does not include frame buffers, the secondoutput image data 422 is synchronized such that its vertical frequencybecomes equal to the vertical frequency of the input image data 412.Therefore, the vertical blanking period of the second output image data422 is set based on the vertical blanking period of the input image data412, and the corresponding vertical synchronization signal is set. Thevertical blanking period of the second output image data 422 ispreferably the same as the vertical blanking period of the input imagedata 412. However, a difference of, for example, several lines isacceptable as long as it is not noticeable on the display screen.

Here, the vertical size of the effective display area 411-2 included inthe second output image data 422 is smaller than the vertical size ofthe effective display area 411-1 included in the first output image data421. Therefore, when the vertical blanking period is set in the secondoutput image data 422 such that its vertical frequency becomes equal tothe vertical frequency of the input image data 412, the blanking periodbecomes longer (i.e., the vertical blanking period becomes longer by aperiod corresponding to an area indicated by a dotted line 430 in FIG.4).

Such an image with a long vertical blanking period does not satisfy thedisplay standard requirements and as a result, the effective displayarea 411-2 cannot be properly displayed on the second display 432.

As illustrated in FIG. 4, when comparing the image data 411 and thefirst output image data 421,

-   -   in the horizontal direction, the display period of the effective        display area 411-1 remains unchanged at H_(t11); and    -   in the vertical direction, the display period of the effective        display area 411-1 remains unchanged at V_(t11).

Also, as illustrated in FIG. 4, when comparing the image data 411 andthe second output image data 422,

-   -   in the horizontal direction, the display period of the effective        display area 411-2 remains unchanged at H_(t21); and    -   in the vertical direction, the display period of the effective        display area 411-2 remains unchanged at V_(t21).

(2) Details of Processes Performed by Devices in Image Display System ofFirst Embodiment (2-1 Details of Processes Performed by Devices)

Next, details of processes performed by the devices in the image displaysystem 140 of the first embodiment are described. FIG. 5 is a drawingillustrating details of processes performed by devices in the imagedisplay system of the first embodiment.

The image generating device 110 generates image data 511 by combining aneffective display area 511-1 of first image data for the first display131 having a resolution of 1920 pixels×1080 pixels and an effectivedisplay area 511-2 of second image data for the second display 132having a resolution of 1280 pixels×720 pixels in the horizontaldirection. The effective display area 511-1 is 1920 pixels×1080 pixels,and the effective display area 511-2 is 1280 pixels×720 pixels.

Also, the image generating device 110 calculates the ratio of thevertical size of the effective display area 511-1 of the first imagedata for the first display 131 to the vertical size of the effectivedisplay area 511-2 of the second image data for the second display 132.Further, the image generating device 110 expands the size of theeffective display area 511-2 of the second image data for the seconddisplay 132 in the vertical direction by a filtering process based onthe calculated ratio. As a result, the image generating device 110obtains an effective display area 512-2.

Next, the image generating device 110 generates image data 512 bycombining the effective display area 511-1 of the first image data forthe first display 131 and the effective display area 512-2 of the secondimage data for the second display 132 in the horizontal direction.

Next, the image generating device 110 outputs the generated image data512 as input image data 513 to the image processing device 120. As aresult, the image processing device 120 obtains the input image data513.

The image processing device 120 generates first output image data 521and second output image data 522 by dividing the input image data 513 inthe horizontal direction.

In the first output image data 521, a horizontal blanking period and ahorizontal synchronization signal are set according to the resolution ofthe first display 131. Here, because the image processing device 120does not include frame buffers, the first output image data 521 issynchronized such that its vertical frequency becomes equal to thevertical frequency of the input image data 513.

Therefore, the vertical blanking period of the first output image data521 is set based on the vertical blanking period of the input image data513, and the corresponding vertical synchronization signal is set. Thevertical blanking period of the first output image data 521 ispreferably the same as the vertical blanking period of the input imagedata 513. However, a difference of, for example, several lines isacceptable as long as it is not noticeable on the display screen.

Accordingly, the effective display area 511-1 is displayed on the firstdisplay 131 at the same frame rate as the input image data 513.

On the other hand, in generating the second output image data 522, theimage processing device 120 generates the effective display area 511-2by reducing the size of the effective display area 512-2 included in theinput image data 513 in the vertical direction by a filtering process.Here, when the vertical size of the effective display area 512-2 afterthe division is reduced in the vertical direction, the number of linesoutput in the same period in the vertical direction is reduced.

Although the effective display area 511-1 of the first output image data521 and the effective display area 511-2 of the second output image data522 have different vertical sizes, their vertical frequencies can bemade equal to each other by adjusting the pixel clock for the secondoutput image data 522.

Therefore, when the vertical blanking period is set in the second outputimage data 522 such that its vertical frequency becomes equal to thevertical frequency of the input image data 513, the vertical blankingperiod can be made shorter than the vertical blanking period set in thesecond output image data 422 illustrated in FIG. 4.

As a result, the image processing device 120 can display the effectivedisplay area 511-2 on the second display 132 at the same frame rate asthe input image data 513.

As illustrated in FIG. 5, when comparing the image data 512 and thefirst output image data 521,

-   -   in the horizontal direction, the display period of the effective        display area 511-1 remains unchanged at H_(t11); and    -   in the vertical direction, the display period of the effective        display area 511-1 remains unchanged at V_(t11).

Also, as illustrated in FIG. 5, when comparing the display period of theeffective display area 512-2 of the image data 512 and the displayperiod of the effective display area 511-2 of the second output imagedata 522,

-   -   in the horizontal direction, the display period remains        unchanged at H_(t21); and    -   in the vertical direction, the display period remains unchanged        at V_(t11).

(2-2 Functional Configuration of Image Processor)

Next, a functional configuration of the image processor 121 isdescribed. FIG. 6 is a drawing illustrating an example of a functionalconfiguration of an image processor of the image processing device ofthe first embodiment. In the case of the image processing device 120 ofthe first embodiment, as illustrated in FIG. 6, the image processor 121includes a controller 601 (control unit and storage unit), an input unit611 (acquisition unit), a divider 612 (dividing unit), and line buffers621 and 622. Also, the image processor 121 includes scalers 631 and 632and output units 641 and 642 (which may be referred to as “imageprocessing unit”). These functional components of the image processor121 may be implemented by a processor (e.g., a CPU) that executes aprogram stored in a memory (e.g., a RAM) to perform processescorresponding to the functions of the functional components. Similarly,the image generator 111 of the image generating device 110 may beimplemented by a processor (e.g., a CPU) that executes a program storedin a memory (e.g., a RAM) to perform a process corresponding to thefunction of the image generator 111.

The controller 601 receives setting data from an external device (notshown) and stores the setting data. Also, the controller 601 reports thestored setting data to the input unit 611, the scalers 631 and 632, andthe output units 641 and 642. For example, the setting data received bythe controller 601 includes:

-   -   the size (1920 pixels×1080 pixels) of the effective display area        511-1,    -   the size (1280 pixels×720 pixels) of the effective display area        511-2, and    -   the expansion ratio (1080 pixels/720 pixels) of the effective        display area 512-2 in the vertical direction.

The input unit 611 sequentially obtains the input image data 513 (forexample, 3400 pixels×1300 pixels) from the image generating device 110at a predetermined pixel clock in units of multiple lines, and outputsthe input image data 513 to the divider 612.

The divider 612 divides the input image data 513 output from the inputunit 611 in the horizontal direction according to display area sizesidentified based on the setting data (divides the input image data 513in the horizontal direction at a position between the effective displayarea 511-1 and the effective display area 512-2). Also, the divider 612generates first intermediate image data 651 and second intermediateimage data 652 by the division, and outputs the first intermediate imagedata 651 and the second intermediate image data 652 to the line buffers621 and 622, respectively.

The line buffers 621 and 622, respectively, store the first intermediateimage data 651 and the second intermediate image data 652 output fromthe divider 612 in units of multiple lines.

The scalers 631 and 632 read the first intermediate image data 651 andthe second intermediate image data 652 stored in the line buffers 621and 622, and perform filtering processes.

In the first embodiment, the image generating device 110 does not expandthe effective display area 511-1 when generating the input image data513. Therefore, the scaler 631 outputs the first intermediate image data651 read from the line buffer 621 to the output unit 641 as firstintermediate image data 661 without reducing its size by the filteringprocess.

On the other hand, in the first embodiment, when generating the inputimage data 513, the image generating device 110 expands the effectivedisplay area 511-2 by a filtering process to obtain the effectivedisplay area 512-2. Therefore, the scaler 632 outputs the secondintermediate image data 652 as second intermediate image data 662 to theoutput unit 642 after performing a filtering process to reduce its sizeby an amount expanded in the vertical direction by the image generatingdevice 110 through the filtering process. Thus, the scaler 632 canreproduce image data equivalent to the second image data for the seconddisplay 132 that is initially generated by the image generating device110.

The output unit 641 generates the first output image data 521 (which hasthe same horizontal frequency and the same vertical frequency as theinput image data 513) based on the first intermediate image data 661output from the scaler 631. The output unit 641 outputs the generatedfirst output image data 521 to the first display 131. As a result, thefirst display 131 can display the effective display area 511-1 suitablefor its resolution (for example, 1920 pixels×1080 pixels) at the sameframe rate as the input image data 513.

On the other hand, the output unit 642 generates the second output imagedata 522 (which has a lower horizontal frequency than and the samevertical frequency as the input image data 513) based on the secondintermediate image data 662 output from the scaler 632. The output unit642 outputs the generated second output image data 522 to the seconddisplay 132. As a result, the second display 132 can display theeffective display area 511-2 suitable for its resolution (for example,1280 pixels×720 pixels) at the same frame rate as the input image data513 without including a non-display area.

(2-3 Image Processing by Image Processing Device)

Next, image processing performed by the image processor 121 of the imageprocessing device 120 is described. FIG. 7 is a flowchart illustratingimage processing performed by the image processor of the imageprocessing device according to the first embodiment.

At step S701, the input unit 611 sequentially obtains the input imagedata 513 from the image generating device 110 in units of multiplelines.

At step S702, the divider 612 divides the input image data 513 in thehorizontal direction to generate the first intermediate image data 651and the second intermediate image data 652.

At step S711, the divider 612 performs post processing such as gammacorrection on the generated first intermediate image data 651, andstores the first intermediate image data 651 in the line buffer 621 inunits of multiple lines.

At step S712, the scaler 631 determines whether to perform a filteringprocess based on the setting data reported from the controller 601. Whenit is determined at step S712 that the filtering process is notperformed (NO at step S712), the process proceeds to step S714.

On the other hand, when it is determined at step S712 that the filteringprocess is performed (YES at step S712), the process proceeds to stepS713. At step S713, the scaler 631 performs the filtering process on thefirst intermediate image data 651.

At step S714, the output unit 641 generates the first output image data521 based on the first intermediate image data 661 output from thescaler 631, and outputs the first output image data 521 to the firstdisplay 131.

On the other hand, at step S721, the divider 612 performs postprocessing such as gamma correction on the generated second intermediateimage data 652, and stores the second intermediate image data 652 in theline buffer 622 in units of multiple lines.

At step S722, the scaler 632 determines whether to perform a filteringprocess based on the setting data reported from the controller 601. Whenit is determined at step S722 that the filtering process is notperformed (NO at step S722), the process proceeds to step S724.

On the other hand, when it is determined at step S722 that the filteringprocess is performed (YES at step S722), the process proceeds to stepS723. At step S723, the scaler 632 performs the filtering process on thesecond intermediate image data 652.

At step S724, the output unit 642 generates the second output image data522 based on the second intermediate image data 662 output from thescaler 632, and outputs the second output image data 522 to the seconddisplay 132.

SUMMARY

As is clear from the above descriptions, the following processes areperformed in the image display system of the first embodiment.

-   -   The image generating device generates an effective display area        (1920 pixels×1080 pixels) of first image data for a first        display. Also, the image generating device generates an        effective display area (1280 pixels×1080 pixels) by expanding        the size of an effective display area (1280 pixels×720 pixels)        of second image data for a second display in the vertical        direction. Further, the image generating device generates input        image data by combining the generated effective display area        (1920 pixels×1080 pixels) and the generated effective display        area (1280 pixels×1080 pixels) in the horizontal direction, and        outputs the input image data to the image processing device.    -   The image processing device obtains the input image data output        from the image generating device.    -   The image processing device generates first intermediate image        data and second intermediate image data by dividing the obtained        input image data in the horizontal direction. Also, the image        processing device performs a filtering process on the second        intermediate image data to reduce the vertical size of the        second intermediate image data and thereby generate second        output image data.

With this configuration, the image display system of the firstembodiment can output the second output image data at a horizontalfrequency lower than that of the input image data even when line buffersare used instead of frame buffers. This in turn makes it possible toreduce the vertical blanking period in the second output image datahaving the same vertical frequency as the input image data.

That is, the image display system of the first embodiment can reduce theblanking period when displaying sets of intermediate image data, whichare obtained by dividing input image data and have different sizes, atthe same frame rate. This in turn makes it possible to display sets ofoutput image data at the same frame rate as input image data withoutincluding a non-display area.

Second Embodiment

In the configuration of the first embodiment, the scaler performs afiltering process to reduce the number of lines of an effective displayarea. On the other hand, in a second embodiment, the scaler performs athinning process to reduce the number of lines of an effective displayarea. Below, differences between the second embodiment and the firstembodiment are mainly described.

<Details of Processes Performed by Devices in Image Display System ofSecond Embodiment> (1) Details of Processes Performed by Devices

First, details of processes performed by devices in an image displaysystem 140 of the second embodiment are described. FIG. 8 is a drawingillustrating details of processes performed by devices in the imagedisplay system of the second embodiment.

The image generating device 110 generates image data 511 by combining,in the horizontal direction, an effective display area 511-1 of firstimage data for the first display 131 having a resolution of 1920pixels×1080 pixels and an effective display area 511-2 of second imagedata for the second display 132 having a resolution of 1280 pixels×720pixels. The effective display area 511-1 is 1920 pixels×1080 pixels, andthe effective display area 511-2 is 1280 pixels×720 pixels.

Also, the image generating device 110 calculates a difference betweenthe vertical size of the effective display area 511-1 of the first imagedata for the first display 131 and the vertical size of the effectivedisplay area 511-2 of the second image data for the second display 132.

Further, the image generating device 110 expands the effective displayarea 511-2 of the second image data for the second display 132 byinserting multiple dummy lines based on the calculated difference andthereby obtains an effective display area 812-2. For example, the imagegenerating device 110 inserts dummy lines by copying adjacent lines.

Next, the image generating device 110 generates image data 812 bycombining the effective display area 511-1 of the first image data forthe first display 131 and the effective display area 812-2 of the secondimage data for the second display 132 in the horizontal direction.

Next, the image generating device 110 outputs the generated image data812 as input image data 813 to the image processing device 120. As aresult, the image processing device 120 obtains the input image data813.

The image processing device 120 generates first output image data 521and second output image data 522 by dividing the input image data 813 inthe horizontal direction.

In the first output image data 521, a horizontal blanking period and ahorizontal synchronization signal are set according to the resolution ofthe first display 131. Here, because the image processing device 120does not include frame buffers, the first output image data 521 issynchronized such that its vertical frequency becomes equal to thevertical frequency of the input image data 813. Therefore, the verticalblanking period of the first output image data 521 is set based on thevertical blanking period of the input image data 813, and thecorresponding vertical synchronization signal is set. The verticalblanking period of the first output image data 521 is preferably thesame as the vertical blanking period of the input image data 813.However, a difference of, for example, several lines is acceptable aslong as it is not noticeable on the display screen.

Accordingly, the effective display area 511-1 is displayed on the firstdisplay 131 at the same frame rate as the input image data 813.

On the other hand, in generating the second output image data 522, theimage processing device 120 reduces the size of the effective displayarea 812-2 included in the input image data 813 in the verticaldirection by removing the dummy lines through a thinning process, andthereby generates the effective display area 511-2. Here, when thevertical size of the effective display area 812-2 after the division isreduced in the vertical direction, the number of lines output in thesame period in the vertical direction is reduced.

Although the effective display area 511-1 of the first output image data521 and the effective display area 511-2 of the second output image data522 have different vertical sizes, their vertical frequencies can bemade equal to each other by adjusting the pixel clock for the secondoutput image data 522.

Therefore, when the vertical blanking period is set in the second outputimage data 522 such that its vertical frequency becomes equal to thevertical frequency of the input image data 813, the vertical blankingperiod can be made shorter than the vertical blanking period set in thesecond output image data 422 illustrated in FIG. 4.

As a result, the image processing device 120 can display the effectivedisplay area 511-2 on the second display 132 at the same frame rate asthe input image data 813.

As illustrated in FIG. 8, when comparing the image data 812 and thefirst output image data 521,

-   -   in the horizontal direction, the display period of the effective        display area 511-1 remains unchanged at H_(t11); and    -   in the vertical direction, the display period of the effective        display area 511-1 remains unchanged at V_(t11).

Also, as illustrated in FIG. 8, when comparing the display period of theeffective display area 812-2 of the image data 812 and the displayperiod of the effective display area 511-2 of the second output imagedata 522,

-   -   in the horizontal direction, the display period remains        unchanged at H_(t21); and    -   in the vertical direction, the display period remains unchanged        at V_(t11).

(2) Functional Configuration of Image Processor

Next, the functional configuration of the image processor 121 isdescribed. FIG. 9 is a drawing illustrating an example of a functionalconfiguration of the image processor of the image processing deviceaccording to the second embodiment. The functional configurationillustrated in FIG. 9 differs from the functional configurationillustrated in FIG. 6 in scalers 931 and 932.

The scalers 931 and 932, respectively, read first intermediate imagedata 901 and second intermediate image data 902 stored in the linebuffers 621 and 622 line by line and perform a thinning process.

In the second embodiment, when generating the input image data 813, theimage generating device 110 does not insert dummy lines into theeffective display area 511-1. Therefore, the scaler 931 outputs thefirst intermediate image data 901 read line by line from the line buffer621 to the output unit 641 as first intermediate image data 911 withoutreducing its size by the thinning process.

As a result, the first display 131 can display the effective displayarea 511-1 suitable for its resolution (for example, 1920 pixels×1080pixels) at the same frame rate as the input image data 813.

On the other hand, in the second embodiment, when generating the inputimage data 813, the image generating device 110 expands the effectivedisplay area 511-2 by inserting dummy lines into the effective displayarea 511-2 and thereby obtains the effective display area 812-2.

Therefore, the scaler 932 performs a thinning process on the secondintermediate image data 902 read line by line from the line buffer 622to remove the dummy lines and reduce its vertical size and therebygenerates second intermediate image data 912.

As a result, the second display 132 can display the effective displayarea 511-2 suitable for its resolution (for example, 1280 pixels×720pixels) at the same frame rate as the input image data 813.

Here, the scaler 932 performs the thinning process to reduce the size ofthe second intermediate image data 902 by an amount expanded by theimage generating device 110 in the vertical direction by inserting thedummy lines, and outputs the resulting image data as the secondintermediate image data 912 to the output unit 642. Thus, the scaler 932can reproduce image data equivalent to the second image data for thesecond display 132 initially generated by the image generating device110.

(3) Image Processing Performed by Image Processing Device

Next, image processing performed by the image processor 121 of the imageprocessing device 120 is described. FIG. 10 is a flowchart illustratingimage processing performed by the image processor of the imageprocessing device according to the second embodiment. FIG. 10 differsfrom FIG. 7 in steps S1001, S1002, S1011, and S1012.

At step S1001, the scaler 931 determines whether to perform a thinningprocess based on the setting data reported from the controller 601. Whenit is determined at step S1001 that the thinning process is notperformed (NO at step S1001), the process proceeds to step S714.

On the other hand, when it is determined at step S1001 that the thinningprocess is performed (YES at step S1001), the process proceeds to stepS1002. At step S1002, the scaler 931 performs the thinning process onthe first intermediate image data 901.

Similarly, at step S1011, the scaler 932 determines whether to perform athinning process based on the setting data reported from the controller601. When it is determined at step S1011 that the thinning process isnot performed (NO at step S1011), the process proceeds to step S724.

On the other hand, when it is determined at step S1011 that the thinningprocess is performed (YES at step S1011), the process proceeds to stepS1012. At step S1012, the scaler 932 performs the thinning process onthe second intermediate image data 902.

SUMMARY

As is clear from the above descriptions, the following processes areperformed in the image display system of the second embodiment.

-   -   The image generating device generates an effective display area        (1920 pixels×1080 pixels) of first image data for a first        display. Also, the image generating device generates an        effective display area (1280 pixels×1080 pixels) by inserting        dummy lines into an effective display area (1280 pixels×720        pixels) of second image data for a second display and thereby        expanding its size in the vertical direction. Further, the image        generating device generates input image data by combining the        generated effective display area (1920 pixels×1080 pixels) and        the generated effective display area (1280 pixels×1080 pixels)        in the horizontal direction, and outputs the input image data to        the image processing device.    -   The image processing device obtains the input image data output        from the image generating device.    -   The image processing device generates first intermediate image        data and second intermediate image data by dividing the obtained        input image data in the horizontal direction. Also, the image        processing device performs a thinning process on the second        intermediate image data to reduce the vertical size of the        second intermediate image data and thereby generate second        output image data.

With this configuration, the image display system of the secondembodiment can output the second output image data at a horizontalfrequency lower than that of the input image data even when line buffersare used instead of frame buffers. This in turn makes it possible toreduce the vertical blanking period in the second output image datahaving the same vertical frequency as the input image data.

That is, the image display system of the second embodiment can reducethe blanking period when displaying sets of intermediate image data,which are obtained by dividing input image data and have differentsizes, at the same frame rate. This in turn makes it possible to displaysets of output image data at the same frame rate as input image datawithout including a non-display area.

Third Embodiment

In the first embodiment, the scaler performs a filtering process toreduce the number of lines of an effective display area; and in thesecond embodiment, the scaler performs a thinning process to reduce thenumber of lines of an effective display area.

On the other hand, in a third embodiment, the scaler shifts the wrappingposition of each line to change the aspect ratio and thereby reduce thenumber of lines of an effective display area. Below, differences betweenthe third embodiment and the first and second embodiments are mainlydescribed.

<Details of Processes Performed by Devices in Image Display System ofThird Embodiment> (1) Details of Processes Performed by Devices

First, details of processes performed by devices in an image displaysystem 140 of the third embodiment are described. FIG. 11 is a drawingillustrating details of processes performed by devices in the imagedisplay system of the third embodiment.

The image generating device 110 generates image data 511 by combining,in the horizontal direction, an effective display area 511-1 of firstimage data for the first display 131 having a resolution of 1920pixels×1080 pixels and an effective display area 511-2 of second imagedata for the second display 132 having a resolution of 1280 pixels×720pixels. The effective display area 511-1 is 1920 pixels×1080 pixels, andthe effective display area 511-2 is 1280 pixels×720 pixels.

Also, the image generating device 110 calculates the ratio of thevertical size of the effective display area 511-1 of the first imagedata for the first display 131 to the vertical size of the effectivedisplay area 511-2 of the second image data for the second display 132.

Further, the image generating device 110 expands the effective displayarea 511-2 of the second image data for the second display 132 in thevertical direction by shifting the wrapping position of each line of theeffective display area 511-2 based on the calculated ratio. As a result,the image generating device 110 obtains an effective display area1112-2.

Next, the image generating device 110 generates image data 1112 bycombining the effective display area 511-1 of the first image data forthe first display 131 and the effective display area 1112-2 of thesecond image data for the second display 132 in the horizontaldirection.

Next, the image generating device 110 outputs the generated image data1112 as input image data 1113 to the image processing device 120. As aresult, the image processing device 120 obtains the input image data1113.

The image processing device 120 generates the first output image data521 and the second output image data 522 by dividing the input imagedata 1113 in the horizontal direction.

In the first output image data 521, a horizontal blanking period and ahorizontal synchronization signal are set according to the resolution ofthe first display 131. Here, because the image processing device 120does not include frame buffers, the first output image data 521 issynchronized such that its vertical frequency becomes equal to thevertical frequency of the input image data 1113. Therefore, the verticalblanking period of the first output image data 521 is set based on thevertical blanking period of the input image data 1113, and thecorresponding vertical synchronization signal is set. The verticalblanking period of the first output image data 521 is preferably thesame as the vertical blanking period of the input image data 1113.However, a difference of, for example, several lines is acceptable aslong as it is not noticeable on the display screen.

Accordingly, the effective display area 511-1 is displayed on the firstdisplay 131 at the same frame rate as the input image data 1113.

On the other hand, when generating the second output image data 522, theimage processing device 120 reduces the vertical size of the effectivedisplay area 1112-2 included in the input image data 1113 by restoringthe wrapping position of each line to its original position and therebygenerates the effective display area 511-2. Here, when the vertical sizeof the effective display area 1112-2 after the division is reduced inthe vertical direction, the number of lines output in the same period inthe vertical direction is reduced.

Although the effective display area 511-1 of the first output image data521 and the effective display area 511-2 of the second output image data522 have different vertical sizes, their vertical frequencies can bemade equal to each other by adjusting the pixel clock for the secondoutput image data 522.

Therefore, when the vertical blanking period is set in the second outputimage data 522 such that its vertical frequency becomes equal to thevertical frequency of the input image data 1113, the vertical blankingperiod can be made shorter than the vertical blanking period set in thesecond output image data 422 illustrated in FIG. 4.

As a result, the image processing device 120 can display the effectivedisplay area 511-2 on the second display 132 at the same frame rate asthe input image data 1113.

As illustrated in FIG. 11, when comparing the image data 1112 and thefirst output image data 521,

-   -   in the horizontal direction, the display period of the effective        display area 511-1 remains unchanged at H_(t11); and    -   in the vertical direction, the display period of the effective        display area 511-1 remains unchanged at V_(t11).

Also, as illustrated in FIG. 11, when comparing the display period ofthe effective display area 1112-2 of the image data 1112 and the displayperiod of the effective display area 511-2 of the second output imagedata 522,

-   -   in the horizontal direction, the display period changes from        H_(t22) to H_(t21) as a result of restoring the wrapping        position to the original position; and    -   in the vertical direction, the display period remains unchanged        at V_(t11).

(2) Functional Configuration of Image Processor

Next, a functional configuration of the image processor 121 isdescribed. FIG. 12 is a drawing illustrating an example of a functionalconfiguration of the image processor of the image processing deviceaccording to the third embodiment. The functional configurationillustrated in FIG. 12 differs from the functional configurationillustrated in FIG. 6 in scalers 1231 and 1232.

The scalers 1231 and 1232, respectively, read first intermediate imagedata 1201 and second intermediate image data 1202 stored in the linebuffers 621 and 622 line by line and perform a process to restore thewrapping position of each line to the original position.

In the third embodiment, the image generating device 110 does not shiftthe wrapping position of each line of the effective display area 511-1when generating the input image data 1113. Therefore, the scaler 1231outputs the first intermediate image data 1201 read line by line fromthe line buffer 621 to the output unit 641 as first intermediate imagedata 1211 without reducing the size of the first intermediate image data1201 by a wrapping position restoring process.

Accordingly, the first display 131 can display the effective displayarea 511-1 suitable for its resolution (for example, 1920 pixels×1080pixels) at the same frame rate as the input image data 1113.

On the other hand, in the third embodiment, when generating the inputimage data 1113, the image generating device 110 expands the effectivedisplay area 511-2 by shifting the wrapping position of each line of theeffective display area 511-2 to obtain the effective display area1112-2.

For this reason, the scaler 1232 reduces the vertical size of the secondintermediate image data 1202 read line by line from the line buffer 622by performing a wrapping position restoring process and therebygenerates second intermediate image data 1212.

As a result, the second display 132 can display the effective displayarea 511-2 suitable for its resolution (for example, 1280 pixels×720pixels) at the same frame rate as the input image data 1113 withoutincluding a non-display area.

Here, the scaler 1232 restores the wrapping position to the originalposition to reduce the size of the second intermediate image data 1202by an amount expanded by the image generating device 110 in the verticaldirection by shifting the wrapping position, and outputs the resultingimage as the second intermediate image data 1212 to the output unit 642.Thus, the scaler 1232 can reproduce image data equivalent to the secondimage data for the second display 132 initially generated by the imagegenerating device 110.

(3) Image Processing Performed by Image Processing Device

Next, image processing performed by the image processor 121 of the imageprocessing device 120 is described. FIG. 13 is a flowchart illustratingimage processing performed by the image processor of the imageprocessing device according to the third embodiment. FIG. 13 differsfrom FIG. 7 in steps S1301, S1302, S1311, and S1312.

At step S1301, the scaler 1231 determines whether to perform a wrappingposition restoring process based on the setting data reported from thecontroller 601. When it is determined at step S1301 that the wrappingposition restoring process is not performed (NO at step S1301), theprocess proceeds to step S714.

On the other hand, when it is determined at step S1301 that the wrappingposition restoring process is performed (YES at step S1301), the processproceeds to step S1302. At step S1302, the scaler 1231 performs thewrapping position restoring process on the first intermediate image data1201.

Similarly, at step S1311, the scaler 1232 determines whether to performa wrapping position restoring process based on the setting data reportedfrom the controller 601. When it is determined at step S1311 that thewrapping position restoring process is not performed (NO at step S1311),the process proceeds to step S724.

On the other hand, when it is determined at step S1311 that the wrappingposition restoring process is performed (YES at step S1311), the processproceeds to step S1312. At step S1312, the scaler 1232 performs thewrapping position restoring process on the second intermediate imagedata 1202.

SUMMARY

As is clear from the above descriptions, the following processes areperformed in the image display system of the third embodiment.

-   -   The image generating device generates an effective display area        (1920 pixels×1080 pixels) of first image data for a first        display. Also, the image generating device generates an        effective display area (850 pixels×1080 pixels) by shifting the        wrapping position of each line of an effective display area        (1280 pixels×720 pixels) of second image data for a second        display and thereby expanding its size in the vertical        direction. Further, the image generating device generates input        image data by combining the generated effective display area        (1920 pixels×1080 pixels) and the generated effective display        area (850 pixels×1080 pixels) in the horizontal direction, and        outputs the input image data to the image processing device.    -   The image processing device obtains the input image data output        from the image generating device.    -   The image processing device generates first intermediate image        data and second intermediate image data by dividing the obtained        input image data in the horizontal direction. Also, the image        processing device performs a wrapping position restoring process        on the second intermediate image data to reduce the size of the        second intermediate image data and thereby generate second        output image data.

With this configuration, the image display system of the thirdembodiment can output the second output image data at a horizontalfrequency lower than that of the input image data even when line buffersare used instead of frame buffers. This in turn makes it possible toreduce the vertical blanking period in the second output image datahaving the same vertical frequency as the input image data.

That is, the image display system of the third embodiment can reduce theblanking period when displaying sets of intermediate image data, whichare obtained by dividing input image data and have different sizes, atthe same frame rate. This in turn makes it possible to display sets ofoutput image data at the same frame rate as input image data withoutincluding a non-display area.

Other Embodiments

In the first through third embodiments described above, each of twoscalers includes a function for processing first intermediate image dataor second intermediate image data and in image processing, the secondintermediate image data is processed.

However, the first intermediate image data may be processed instead ofthe second intermediate image data. Further, both of the firstintermediate image data and the second intermediate image data may beprocessed.

Also, in the first through third embodiments, one of the first outputimage data and the second output image data is output to each of twodisplays. However, the number of displays is not limited to two, and theimage display system may be configured such that one of first throughNth sets of output image data is output to each of N displays (N is aninteger greater than or equal to three).

In the first through third embodiments, it is assumed that theresolution of the first display 131 is 1920 pixels×1080 pixels and theresolution of the second display 132 is 1280 pixels×720 pixels. However,the resolution of the first display 131 and the resolution of the seconddisplay 132 are not limited to these examples. In the first throughthird embodiments, the resolution of the first display 131 is higherthan the resolution of the second display 132. However, the resolutionof the second display 132 may be higher than the resolution of the firstdisplay 131. In this case, the scaler 631/931/1231 performs a filteringprocess, a thinning process, or a wrapping position restoring process onthe first intermediate image data 651/901/1201.

In the first through third embodiments, the image generating device 110calculates a ratio or a difference and then expands the effectivedisplay area of the second image data in the vertical direction.However, the image generating device 110 may be configured to read apre-calculated ratio or difference and expand the effective display areaof the second image data in the vertical direction.

Further, in the first through third embodiments, it is assumed that theimage display system 140 is installed in the vehicle 100. However, theimage display system 140 may be used for other applications.

An image processing device, an image processing method, and an imagedisplay system according to embodiments of the present invention aredescribed above. However, the present invention is not limited to theabove-described embodiments, and modifications may be made withoutdeparting from the scope of the present invention. For example, theabove-described configurations may be combined with other elementsdepending on the applications of the present invention.

What is claimed is:
 1. An image processing device, comprising: a memoryconfigured to store a program; and a processor configured to execute theprogram stored in the memory to perform a process including obtaininginput image data that is generated by combining, in a horizontaldirection, an effective display area of first image data and aneffective display area obtained by expanding an effective display areaof second image data in a vertical direction, a vertical size of thesecond image data being smaller than a vertical size of the first imagedata, and generating second output image data corresponding to thesecond image data by reducing a vertical size of second intermediateimage data among first intermediate image data and the secondintermediate image data, which are obtained by dividing the input imagedata in the horizontal direction, based on a vertical size of theeffective display area of the second image data.
 2. The image processingdevice as claimed in claim 1, further comprising: a line bufferconfigured to store the second intermediate image data in units ofmultiple lines, wherein in the generating, the second intermediate imagedata is read from the line buffer and the vertical size of the secondintermediate image data is reduced.
 3. The image processing device asclaimed in claim 2, wherein the vertical size of the second intermediateimage data is reduced by a filtering process.
 4. The image processingdevice as claimed in claim 3, wherein the input image data is generatedby combining, in the horizontal direction, the effective display area ofthe first image data and the effective display area obtained byexpanding the vertical size of the effective display area of the secondimage data by the filtering process.
 5. The image processing device asclaimed in claim 2, wherein the vertical size of the second intermediateimage data is reduced by a thinning process.
 6. The image processingdevice as claimed in claim 5, wherein the input image data is generatedby combining, in the horizontal direction, the effective display area ofthe first image data and the effective display area obtained byexpanding the vertical size of the effective display area of the secondimage data by inserting dummy lines.
 7. The image processing device asclaimed in claim 2, wherein the vertical size of the second intermediateimage data is reduced by shifting a wrapping position of each line. 8.The image processing device as claimed in claim 7, wherein the inputimage data is generated by combining, in the horizontal direction, theeffective display area of the first image data and the effective displayarea obtained by expanding the vertical size of the effective displayarea of the second image data by shifting the wrapping position of eachline.
 9. The image processing device as claimed in claim 1, wherein theprocess further includes dividing the input image data in the horizontaldirection.
 10. The image processing device as claimed in claim 9,wherein the process further includes identifying a horizontal size basedon setting data for the input image data; and in the dividing, the inputimage data is divided in the horizontal direction based on theidentified horizontal size.
 11. The image processing device as claimedin claim 9, further comprising: a storage configured to store ahorizontal size according to which the input image data is divided inthe horizontal direction, wherein in the dividing, the input image datais divided in the horizontal direction based on the horizontal sizestored in the storage.
 12. The image processing device as claimed inclaim 1, wherein the process further includes generating first outputimage data corresponding to the first image data based on the firstintermediate image data.
 13. An image processing method, comprising:obtaining input image data that is generated by combining, in ahorizontal direction, an effective display area of first image data andan effective display area obtained by expanding an effective displayarea of second image data in a vertical direction, a vertical size ofthe second image data being smaller than a vertical size of the firstimage data; and generating, by a processor, second output image datacorresponding to the second image data by reducing a vertical size ofsecond intermediate image data among first intermediate image data andthe second intermediate image data, which are obtained by dividing theinput image data in the horizontal direction, based on a vertical sizeof the effective display area of the second image data.
 14. The imageprocessing method as claimed in claim 13, wherein the vertical size ofthe second intermediate image data is reduced by a filtering process.15. The image processing method as claimed in claim 13, wherein thevertical size of the second intermediate image data is reduced by athinning process.
 16. The image processing method as claimed in claim13, wherein the vertical size of the second intermediate image data isreduced by shifting a wrapping position of each line.
 17. The imageprocessing method as claimed in claim 13, further comprising: generatingfirst output image data corresponding to the first image data based onthe first intermediate image data.
 18. An image display system,comprising: an image generating device configured to generate inputimage data by combining, in a horizontal direction, an effective displayarea of first image data and an effective display area obtained byexpanding an effective display area of second image data in a verticaldirection, a vertical size of the second image data being smaller than avertical size of the first image data; and an image processing deviceconfigured to generate second output image data corresponding to thesecond image data by reducing a vertical size of second intermediateimage data among first intermediate image data and the secondintermediate image data, which are obtained by dividing the input imagedata in the horizontal direction, based on a vertical size of theeffective display area of the second image data.
 19. The image displaysystem as claimed in claim 18, wherein the image processing device isconfigured to generate first output image data corresponding to thefirst image data based on the first intermediate image data, and theimage display system further includes: a first display device configuredto display the first output image data, and a second display deviceconfigured to display the second output image data.